The present invention, in some embodiments thereof, relates to a dynamic multi-sensor and multi-machine calibration method.
Computer and robotic applications that include some level of interaction with the physical environment, require estimating and sharing the 2D or 3D coordinates of objects detected in the given environment. When more than one sensor or actuator is involved, it's critical to have these coordinates translated to a relative position with respect to the receiving device or sensor. For example, if an object is detected after analysing an output image from a depth sensor, in order to have a robot grasp this object, those coordinates must be translated to a relative position with respect to the robot itself as opposed to the camera that detected it. When multiple sensors and robots are combined, obtaining these equivalent coordinates becomes very complex.
Many industrial applications of robotic machines include robotic machines working together as a robotic team. These teams of robotic machines include machines of multiple designs, manufacturers and generations and are many times assisted by external sensors to determine the location of objects in the environment. In order to have each machine understand where a given object is located, the objects' coordinates must be converted to a relative position with regards to of the participating machines. In static environments such as the industrial one described, this requires some setup effort to determine the relative position of each sensor and robot in the system with respect to each other. After this initial effort and considering that the base of the robots and the sensors are in a fixed position, the initial data is used to make the required calculations for sharing coordinate data between robots and sensors.
In a less static environment, an initial setup effort to estimate these relative positions is not enough as both robots' bases and sensors might change positions in an untraceable way, and therefore the relative positions between them will not always be the same.
While requiring setup effort, there are many methods to translate coordinates between devices in the static scenario described above. For a dynamically changing scenario, where sensors and robots change their relative position with regards to each other, the problem is more complicated and there are no practical and dynamical methods available that solve this in a simple and timely manner.
In other embodiments of the current invention, the methods described above are used as a system to assist physically impaired patients who can demand actions from a robot combining one or more gesture mechanisms: Eye gaze, voice, gestures, EEG signals, touch, and others. A robot is set to assist a paralyzed or weak patient in manipulating or picking objects around a scene. A method is defined to enable such patients to control a robotic arm through gaze. A camera pointed towards an end-user's eye is able to track the direction the pupil is directed to while one or more external sensors is able to detect the object in the scene that the user is observing. A special human-machine interface system is able to translate gaze movements into requests by the end-user and translate this into actions that the robot can perform on scene objects.